Effects of increasing modalities in understanding three simultaneous speeches with two microphones
نویسندگان
چکیده
This paper reports effects of increasing modalities in understanding three simultaneous speeches with two microphones. This problem is difficult because the beamforming technique adopted for a microphone array needs at least four microphones, and because independent component analysis adopted for blind source separation needs at least three microphones. We investigate four cases; monaural (one microphone), binaural (two microphones), binaural with independent component analysis (ICA), and binaural with vision (two microphones and two cameras). The performance of word recognition of three simultaneous speeches is improved by adding more modalities, that is monaural, binaural, and binaural with vision.
منابع مشابه
Separating three simultaneous speeches with two microphones by integrating auditory and visual processing
This paper addresses the problem of automatic recognition of three simultaneous speeches with two microphones, that is, that of sound source separation where the number of sound sources is greater than that of microphones. The approach used is the direction-pass filter, which is implemented by hypothetical reasoning on the interaural phase difference (IPD) and interaural intensity difference (I...
متن کاملImprovement of three simultaneous speech recognition by using AV integration and scattering theory for humanoid
This paper presents improvement of recognition of three simultaneous speeches for a humanoid robot with a pair of microphones. In such situations, sound separation and automatic speech recognition (ASR) of the separated speech are difficult, because the number of simultaneous talkers exceeds that of its microphones, the signal-to-noise ratio is quite low (around -3 dB) and noise is not stable d...
متن کاملUnderstanding Three Simultaneous Speeches
Understanding three simultaneous speeches is proposed as a challenge problem to foster artificial intelligence, speech and sound understanding or recognition, and computational auditory scene analysis research. Automatic speech recognition under noisy environments is attacked by speech enhancement techniques such as noise reduction and speaker adaptation. However, the signal-to-noise ratio of s...
متن کاملChallenge Problem for Computational Auditory Scene Analysis: Understanding Three Simultaneous Speeches
Understanding three simultaneous speeches is proposed as a challenge problem to foster arti cial intelligence, speech and sound understanding or recognition, and computational auditory scene analysis research. Automatic speech recognition under noisy environments is attacked by speech enhancement techniques such as noise reduction and speaker adaptation. However, the signal-to-noise ratio of sp...
متن کاملMultimedia Annotation: Comparability of Gloss Modalities and their Implications for Reading Comprehension
This study compared the effects of two annotation modalities on the reading comprehension of Iranian intermediate level EFL learners. The two experimental groups under study received treatment on 10 academic L2 reading passages under one of two conditions: One group received treatment on key words in the reading passages through a multimedia environment providing textual annotations. The second...
متن کامل